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Background: Poor coordination between levels of care plays a central role in determining 
the quality and cost of health care. To improve patient coordination, systematic structures, 
guidelines, and processes for creating, transferring, and recognizing information are needed to 
facilitate referral routines. 

Methods: Prospective observational survey of implementation of electronic medical record 
(EMR)-supported guidelines for surgical treatment. 

Results: One university clinic, two local hospitals, 3 1 municipalities, and three EMR vendors 
participated in the implementation project. Surgical referral guidelines were developed using 
the Delphi method; 22 surgeons and seven general practitioners (GPs) needed 109 hours to 
reach consensus. Based on consensus guidelines, an electronic referral service supported by a 
clinical decision support system, fully integrated into the GPs' EMR, was developed. Fifty- five 
information technology personnel and 563 hours were needed (total cost 67,000 £) to implement 
a guideline supported system in the EMR for 139 GPs. Economical analyses from a hospital and 
societal perspective, showed that 504 (range 401-670) and 37 (range 29^49) referred patients, 
respectively, were needed to provide a cost-effective service. 

Conclusion: A considerable amount of resources were needed to reach consensus on the surgical 
referral guidelines. A structured approach by the Delphi method and close collaboration between 
IT personnel, surgeons and primary care physicians were needed to reach consensus. 
Keywords: hospital referrals, surgery, patient pathways, process health care assessment, 
electronic medical record 

Introduction 

Busy hospital clinics and increased waiting time for surgical treatment have made it 
necessary to develop new referral routines and to improve the patient flow and coor- 
dination between primary and secondary care. Poor coordination between different 
levels of care plays a central role in determining health care quality and cost. 13 To 
improve coordination, systematic structures, guidelines (clinical decision support 
[CDS] systems), and processes for creating, transferring, and recognizing information 
are needed to facilitate primary-secondary care communication and referral routines. ' 
Improved referral routines have the potential to decrease waiting time and to decrease 
the workload for hospital clinics. 4-7 CDS systems can be used as a tool to improve 
referral quality and the coordination of care. CDS systems provide clinicians with 
patient-specific assessment tools or guidelines to aid their clinical decision-making 
and have been shown to improve health care quality. 8 - 9 CDS systems improve prescrib- 
ing practices, reduce serious medication errors, enhance the delivery of preventive 
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care services, improve adherence to guidelines, and result 
in lasting improvements in clinical practice. 10-13 

In this research project we focused on referrals to out- 
patient surgery. Outpatient surgery is a cost-effective solu- 
tion for certain surgical patients because it decreases the 
time between the referral and the surgery and subsequently 
improves the patient flow. 5,14-19 Furthermore, outpatient sur- 
gery decreases the burden of internal hospital logistics by 
decreasing the number of administrative personnel involved 
in the referral process. 14 ~ 1 ' 

In 2007, we initiated an outpatient surgical referral 
program, aiming to decrease waiting times and improve the 
patient flow at the outpatient clinics and operational theatres. 
We wanted to implement a CDS-supported referral system 
in the general practitioners' (GPs') electronic medical record 
(EMR) and thereby improve surgical referral quality, improve 
the coordination of care, and increase the rate of outpatient 
surgery (one stop surgery). The objective of this observational 
trial was to describe the Delphi method and resources needed 
(money, time, and personnel) to develop and implement a 
CDS-supported surgical referral system in the GPs' EMR. 

Methods 

This prospective observational study took place between 
June 2007 and June 201 1 and was divided into three main 
phases: 1) reaching guideline consensus on the content of 
a CDS-supported referral service; 2) developing the CDS- 
supported referral software; and 3) large-scale implemen- 
tation of an EMR-integrated referral service. The use of 
resources (surgeons, GPs, administrative staff, and technol- 
ogy) was prospectively registered. The primary objective 
of the survey was to describe the primary care physician- 
surgeon consensus process by the Delphi method. The sec- 
ondary objective was to perform an estimate of the number 
of referrals needed to establish a cost-effective referral ser- 
vice (ie, cost of consensus and implementation = potential 
cost savings by improved referral routines). 

Research setting 

The Department of Digestive Surgery is divided into 
clinics located at three different hospitals. All of these 
clinics perform the surgical procedures included in the trial 
(pilonidal sinus: Bascom plasty; inguinal hernia: Lichtenstein 
repair; gallstone disease: laparoscopic cholecystectomy). In 
the county of Troms, there are 31 municipal GP practices 
with 139 GPs. All of the GP practices are connected to the 
Norwegian National Health Network, which sends referrals 
electronically to hospitals. 



Patients 

Patients were included in the analyses if they were diagnosed 
and referred to hospital by their GP for an inguinal hernia, 
umbilical hernia, sinus pilonidalis, or gallstone disease 
requiring surgical treatment. 

Consensus of a guideline-supported 
referral system 

A modified Delphi approach was used, 21 divided into the 
following phases: generation of referral recommendations, 
a panel meeting of senior surgeons, a first consensus round 
with all surgeons (review results), a second consensus 
round with all surgeons (ratings), and the evaluation of the 
consensus. Relevant evidence regarding the different condi- 
tions was presented, and all of the participants in the process 
were asked to include only the "need to know" information 
about the surgical procedure. This information includes: 

• Disease-specific guidelines: These guidelines 
included signs and symptoms, surgical indications 
and contraindications, blood samples, and radiological 
examinations needed prior to surgery (laparoscopic 
cholecystectomy) . 

• Standardized referral forms: The referral forms included 
information needed for anesthesia (mandatory), 
information needed for surgeons (mandatory), and 
additional information. 

• Patient information: This information included the time 
of surgery, the surgical procedure, the instructions for the 
patient to follow after surgery, and the possible complica- 
tions of surgery. 

After we reached internal hospital department consensus, 
the referral templates and guidelines were discussed with a 
panel of three experienced GPs. The GPs provided comments 
and suggestions for further improvement of the referral 
forms, disease-specific guidelines, and patient information. 
All changes suggested by the GPs were then presented and 
accepted by the head of the surgical department. 

All of the necessary resources (hours) used to reach 
consensus were prospectively registered. 

Development of the CDS-supported 
EMR referral software 

Two EMR companies were involved in the development, 
piloting, and implementation of the CDS solution. A third 
company was responsible for the software solution making 
communication between EMR 1 and EMR 2 possible. The 
development and implementation of the technology were 
adjusted to incorporate the features of CDS systems that have 
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been proven effective for changing clinical practice. 20 All of 
the necessary resources were prospectively registered. 

Referral piloting and large-scale 
implementation 

A primary care office was chosen as the pilot office. The 
CDS-supported referral software was discussed with the 
GPs, and improvements were made after receiving feedback 
from the GPs. Test referrals were sent from the GP office to 
the hospital until all of the technological problems had been 
resolved. Information technology (IT) experts integrated 
the referral software into the GPs' EMR. The software was 
fully integrated into the GPs' EMR and triggered by the 
corresponding international classification of primary care 
(ICPC) codes. 

Comparison of referral cost-effectiveness 

Two referral routes were compared: a) the traditional referral 
route where all patients are referred by the GP to the outpa- 
tient clinic and the outpatient clinic surgeon then refers the 
patients to day case surgery; and b) The GP refers directly 
to day case surgery (Figure 1). The sensitivity analysis used 
both a societal and a hospital perspective. The referral costs 
included administrative hospital costs, the examination in the 
surgical outpatient department (surgeon time, nurse time), 
the surgical treatment (surgeon time, anesthetist time, and 
nurse time), the unused surgical outpatient capacity caused by 
incorrect referrals (both for new and traditional service), and 
the patients' travel costs. All of the cost estimates (Table SI) 
were created separately and compared. We assumed that 1 0% 
of the one-stop patients would be denied surgery because of 
inconsistent referral information and that 20% of the referred 
patients were on sick leave at the time of referral. The number 
of patients needed to break even (the threshold, ie, the point at 
which the cost savings from a one stop surgery (OSS) referral 
service equal the cost of implementation) was estimated. This 
value was obtained using the following equation: 

Potential savings of the referral service 

= Cost of the Delphi process + Cost of software 
development + Cost of implementation. (1) 

We defined a potential decrease in waiting time as "total 
waiting time - intrahospital waiting time." Prehospital waiting 
time was defined as the time between the referral and the first 
consultation at the surgical outpatient clinic. Data from the 
national waiting list register was extracted ( http : / /www, fritt- 
sykehusvalg.no/ start/ ) to perform this analysis. Intrahospital 




Outpatient clinic 

■ Clinical examination 

■ Intrahospital referrals 
outpatient surgery 



Outpatient surgery 
■ Surgical procedure 



General practitioner 

■ CDS referral one-stop 
outpatient surgery 



I 



Outpatient surgery 

• Surgical procedure 

• Patients examined by the 
surgeon day of surgery 



Figure I Principle of a guideline-supported surgical referral system to outpatient 
surgery. 

Notes: The aim is to decrease time spent on intrahospital logistics, by omitting 
the outpatient clinical examination and intrahospital referral to day case surgery. 
The traditional referral pathways (upper arrow) may be decreased by high-quality 
GP referrals directly to outpatient surgery (lower arrow). 
Abbreviations: CDS, clinical decision support; GP, general practitioner. 

waiting time was defined as the time between the first con- 
sultation at the outpatient clinic and the surgical treatment. 
The NOMESCO (Nordic Medico-Statistical Committee) 
Classification of Surgical Procedures codes were extracted 
from the hospital's EMR database to perform this analysis. 
To estimate the potential frequency of patients referred by 
the new service, consecutive referrals (an 18-month time 
interval) for sinus pilonidalis (International Classification 
of Diseases [ICD] 10: L05.0 and L05.9), gallstone disease 
(ICD 10: K80.2), inguinal hernia (ICD 10: K40.0, K40.2, 
K40.9), and umbilical hernia (ICD 10: K42.0, K42.9) were 
reviewed by two authors (Knut Magne Augestad [KMA] and 
Rolv-Ole Lindsetmo [ROL]). Cost data were converted from 
Norwegian kroner (NOK) into British pounds at the rate of 
1 £ =9.39 NOK (based on the exchange rage according to 
the Norwegian National Bank on June 27, 2012). The BMJ 
guidelines for health-related economic evaluations were 
followed. 22 

Results 

The Delphi process 

All of the surgeons approved the guidelines, and consensus 
was reached after a 3-month period. Overall, 1 09 hours were 
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needed to reach guideline consensus, to an estimated cost of 
10,800 £ (Tables 1 and 2). 

Development and implementation 
of the referral software 

IT personnel (n=24) from three different commercial vendors 
participated in the development of the CDS software, and 
the overall cost of the software development was estimated 
at 32,652 £. A total of 140 hours were needed to pilot and 
implement the software for 139 GPs, at a total cost of 
23,581 £. The total cost of work phases 1 to 3 was 67,075 £ 
(Table 3). 

Main effects of new referral routines 

We reviewed 700 referrals (ie, 18 months consecutive 
referrals); 189 (27%) were classified as well-suited for the 
referral service. The median waiting time to the examina- 
tion in the outpatient clinic was 84 days, and the median 
time from the examination until the outpatient surgery was 
101 days (ie, the overall median waiting time was 1 85 days). 
The estimated cost savings from the referral service 
were 133 £ per patient from the hospital perspective and 
18,909 £ from the societal perspective (Table 4). 

A tornado chart was employed to analyze the uncertainties 
of referral implementation cost. Small variations in the cost 
of CDS technology development (phase 2) will have large 
effects on the overall cost compared with the other cost ele- 
ments we analyzed (Figure 2). 



Discussion 

Summary of findings 

To our knowledge, this survey is the first to address 
the time, resources, and funding needed to develop and 
implement an electronic hospital referral service supported 
by a CDS system. The decision support tool was fully 
integrated into the GPs' EMR and was triggered by the 
referring ICPC code. A total of 84 persons (surgeons, GPs, 
administrative personnel, and health IT personnel) and 550 
hours (from the beginning of the project) were needed to 
implement the CDS referral software in the EMR for 139 
GPs. The overall cost of the surgical Delphi consensus 
process, the development of the CDS technology, and the 
implementation of the technology was 67,000 £. A sensitiv- 
ity analysis showed that unexpected technology obstacles 
had the greatest impact on the overall cost of CDS imple- 
mentation. Finally, we estimated the effects of an integrated 
CDS referral system for patients and the health care system. 
This surgical referral system has the potential to reduce 
the waiting time for surgery by 101 days. Of the referrals 
for selected surgical conditions, 27% were well-suited for 
a one-stop surgical service, which means that the service 
was cost-effective (from the hospital perspective) after 
504 CDS-supported referrals. Because of the decreased 
waiting time for surgical treatment, the length of sick leave 
will decrease, resulting in societal savings of approximately 
1,809 £ for each patient who is referred to an expedited 
one-stop surgical service. 



Table 1 The Delphi approach of referral consensus for 


surgeons and GPs 








Implementation 


Description 


Responsible 


Participants (n) 


Hours 


Cost (£) 


phase 












1 


Analysis of referral routines, 


PM and senior surgeon 


2 


20 


2,040 




review of the literature 










2 


Surgical team plenum discussion 


All surgeons 


22 


22 


2,244 


3 


Development of CDS 1 .0 


PM 


2 


10 


1,020 






Senior surgeon 








4 


Feedback on CDS 1 .0 from 


Expert surgeons 


3 


9 


918 




expert surgeons 










5 


Development of CDS 2.0 


PM and senior surgeon 


2 


5 


510 


6 


Presentation of CDS 2.0 to all 


PM 


1 


2 


204 




surgeons 










7 


Plenum discussion 


All surgeons 


22 


22 


2,244 


8 


Development of CDS 3.0 


PM and senior surgeon 


2 


5 


510 


9 


Presentation of CDS 3.0 to 


PM and GPs 


3 


3 


306 




general practitioners 










10 


Final approval 


Chief surgical department 


1 


2 


204 


1 1 


Presentation of CDS 3.0 to 


PM and IT personnel 


3 


3 


321 




EMR companies 










12 


Presentation of CDS 3.0 to the 


PM and IT personnel 


3 


3 


321 




IT department 










Total 






29 


109 


10,842 



Abbreviations: CDS, clinical decision support; EMR, electronic medical record; GP, general practitioner; IT, information technology; PM, project manager. 
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Table 2 Example of consensus guidelines for hernia surgery 
referral 

Consensus guidelines for referring an inguinal hernia for 
surgical treatment 

1 . Actual disease (free text) 

2. Previous anesthesia complications (free text)? 

3. Heart function (scroll bar): 

No valvular disease, no coronary heart disease 

Asymptomatic valvular or coronary heart disease (NYHA I and/or 

EF 40%-S0%) 

Light valvular disease and or coronary heart disease induced by 
activity (NYHA 2-3 and/or EF 30%-40%) 

Valvular heart disease and or coronary heart disease with symptoms 
at rest (NYHA 4 or EF <30%) 

If NYHA > 2: refer for a cardiology consultation before surgery 

4. Lung function (scroll bar): 
Normal, no dyspnoea (FEVI >80%) 

Lightly reduced, dyspnea with high physical exercise (FEVI 
80%-60%) 

Moderately reduced, dyspnea with low physical exercise (FEVI 
60%-40%) 

Severely reduced, dyspnea at rest (FEVI <40%) 

If moderately or severely reduced lung function: refer for a lung 

function test before surgery 

5. Renal function (scroll bar): kidney failure/kidney disease? 

6. Other known diseases or risk factors (free text)? 

7. Pharmacology and medicines (yes/no): anti-diabetics, anti- 
arrhythmia, steroids, anti-asthmatics, immunosuppressant, anti- 
coagulants 

8. Allergies (yes/no and free text)? 

9. Previous abdominal surgery (free text)? 
10. Patient weight/height (scroll bar)? 

I I . Symptoms and signs of inguinal hernia (scroll bar): how often the 
hernia is present, size of the hernia, duration (in months) of hernia 
disease, pain when lifting, sick leave due to the hernia, frequency 
of pain. 

12. Clinical examination (yes/no): Palpation of the inguinal canal: 
palpable hernia when coughing, testicle in scrotum, reducibility of 
the hernia 

1 3. Supplementary information (free text). 

14. Has the patient received information about the possible 
complications of the surgery (yes/no)? 

Notes: Ten minutes were needed to complete the form. Most questions were 
answered with scroll bars or yes/no answers. Similar referral forms were developed 
for gallstone disease, umbilical hernia, and sinus pilonidalis. 

Abbreviations: EF, ejection fraction; FEVI, forced expiratory volume in I second; 
NYHA, New York Heart Association classification. 

Comparison with the existing literature 

Effective communication is essential for coordinating 
health care; however, patient care is often compromised by 
poor communication between primary and secondary care 
services. 23 25 Poor coordination between primary and second- 
ary care services contributes to avoidable patient morbidity 
and mortality. 26 Thus, effective communication between 
primary care physicians and hospital specialists regarding 
patient referrals, consultations, and treatment is necessary 



to improve patient outcomes and physician satisfaction. 27 
Poor communication plays a central role in determining 
health care quality and cost, and systematic structures 
and tools (like CDS systems) are needed to facilitate primary- 
secondary care communication. 13 

To address the deficiencies in communication and the 
coordination of care, health care is increasingly turning to 
CDS systems, which provide clinicians with patient-specific 
assessments or recommendations to aid in their clinical 
decision-making and process management. 20 CDS systems 
have been shown to reduce errors and improve the coordination 
of care. 20 Several features of CDS systems are associated with 
improvements in clinical practice (automatic provision, inte- 
gration with the EMR, and provision of recommendations). 
Of these features, automatic provision of decision support as 
part of the clinician's workflow is the single most important 
feature, with a reported odds ratio of 1 05 . 20 - 28 In this project, we 
focused on similar CDS features associated with improvements 
in clinical practice. When we designed the user interface of the 
CDS, the CDS was triggered by the referral ICPC code, thus 
becoming an automatic part of the GPs' referral workflow. 

The rate of referrals has been growing rapidly, with 
potential implications for health care spending. 29 There are 
no widely accepted guidelines for referring patients. Despite 
the push toward evidence-based decision-making, the refer- 
ral of patients is driven primarily by the practice patterns of 
physicians, not by the patient case mix. Patient referrals can 
profoundly change the nature and cost of the care that patients 
receive. We need to understand the benefits and risks of spe- 
cific referral patterns, as we do with any intervention. 303 ' 

Strengths and limitations 

This trial has some strengths. First, many trials have evalu- 
ated the clinical effects of CDS systems; however, few of 
these trials have assessed the effects of the systems on the 
coordination of care between different service levels of health 
care. Second, most studies that have assessed referral systems 
have been conducted without a fully integrated EMR system 
implemented in primary and secondary care. Although two 
separate EMR vendors support our EMR systems, they are 
fully integrated in the entire health care system, with refer- 
rals and text messages sent electronically between the two 
systems. This integration increases the potential of a CDS- 
supported referral service because it offers greater flexibility 
to adjust the CDS system to new routines or new referral 
patterns. The CDS referral system can be updated from one 
central server located at the university hospital (ie, no techni- 
cal personnel have to travel to the GPs' offices). 
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There are also limitations to this study. First, the 
long-term implementation and routine use of CDS systems 
remains challenging. In a recently published review of 
randomized trials on decision support systems, only 18% 
discussed long-term implementation and long-term use. 32 At 
the present time, we have not succeeded in fully incorporating 
the referral service into routine use, a problem that has also 
been reported in other trials. 33 34 We acknowledge that there 
is a need to improve the GP reimbursement policy because 
of the increase in the referral workload. Second as a quality 
improvement project, there are certain limitations that are 
inherent to our work. 

The economic analysis has several limitations. There are 
uncertainties attached especially with regard to the propor- 
tion of patients that can return faster back to work caused by 
decreased waiting times and the proportion of patients who 
are rejected because of improper clinical information in the 



referral. These uncertainties must be assessed using future 
prospective surveys. 

Implications for patients 
and decision-makers 

Higher-quality surgical referrals will have great implica- 
tions for patients. We have analyzed the implications of 
a standardized surgical referral service, which produces a 
shorter waiting time before surgical treatment and subse- 
quently a decrease in sick leave. For other conditions, high- 
quality referrals and closed communication loops improve 
the coordination of care. This means that there will be fewer 
hospital admissions, fewer radiologic exams, fewer blood 
samples, and more informed health care providers offering 
better continuity of care. 1 - 3 ' 24,25 These aspects are especially 
important for patients with multimorbidity and chronic 
diseases, for whom new strategies are needed to coordinate 



Table 3 Referral software development and implementation 



Implementation phase 



Description 



Responsible 



Hours 



Cost (£) 



Technology development 
I 

2 
3 



Project planning/management 

Developing the EMR CDS user interface 
and adopting it to existing standards 
Presenting the suggested solution to users 



Developing the technological CDS EMR 
solution 

Developing the CDS triggers from the 
ICPC referral codes 
Developing the interactor client* 
Piloting the system in an EMR laboratory 



Total technology development 
Piloting GP-to-hospital referrals 



8 
9 
10 
I I 
12 
13 
14 

15 

Large scale implementation 

16 

Cost estimates 

Cost of piloting/implementation 

Cost of technology/piloting/implementation 

Cost of Delphi/technology/piloting/implementation 



Installing the pilot software in GP offices 
Holding discussions with the GPs 
Testing sending referrals 
Holding discussions with office personnel 
Tracking "lost" referrals 
Correcting technological problems 
Adjusting the CDS after receiving GP 
feedback 

Adjusting the CDS after receiving 
administrative feedback 

Installing the software for 1 39 GPs 



PM 

IT personnel 
PM 

IT personnel 
PM 

IT personnel 
surgeons/GPs 
IT personnel 

IT personnel 

IT personnel 
IT personnel 
PM 



IT personnel 
PM 

IT personnel 
PM 

IT personnel 
IT personnel 
IT personnel 
PM 

IT personnel 
PM 

IT personnel 



2 
2 
30 

4 

4 

4 
4 

24 

2 
6 
6 
3 

10 
10 

2 



31 
55 
84 



38 
16 
30 

32 

60 

80 
80 

314 

10 

5 

20 
10 
30 
50 
10 



57 

140 
454 
563 



3,406 
1,712 

2,550 

3,424 

6,420 

8,560 
6,580 

32,652 

1,250 

625 

2,500 

410 

3,750 

6,300 

1,250 

625 

6,871 

23,581 
56,233 
67,075 



Notes: ^Interactor client software that facilitates communication between two different EMR systems in the Norwegian Health Network. EMR system #1 is located in GP 
offices and EMR system #2 is located in the university hospital. 

Abbreviations: CDS, clinical decision support; EMR, electronic medical record; GP, general practitioner; ICPC, international classification of primary care; IT, information 
technology; PM, project manager. 
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Table 4 Estimates of time and cost savings of the new referral routines 



Variable 



Perspective 



Explanation 



Cost/patient (£) Sensitivity analyses 



Traditional referrals routines 

In-hospital logistics H 

Senior surgeon referral appraisal H 

Hospital travel for outpatient appointment H 

In-hospital logistics H 

Hospital travel for outpatient surgery H 

Outpatient surgery H 
Total hospital cost 
Range 

Sick leave a S 

Total cost 
Range 

Guideline-supported referral routines 

Hospital travel H 
Outpatient surgery denied because of H 
inconsistent referrals 8 



Increased frequency of surgical consultations" H 

Outpatient surgery b H 

Total hospital cost H 
Range 

Sick leave* S 

Total cost S 
Range 

Referrals needed to provide a cost-effective service 

Hospital savings/pt (range) 

Hospital refferal C/E threshold n (range) 

Societal savings/pt (range) 

Societal referral C/E threshold n (range) 



30 min/referral 

IS min/referral 

Mean hospital travel 

30 min/referral 

Mean/travel 

Mean three conditions 



1 85 days before 
28 days after 



30 surgeries (10%) 
Lichtenstein I 1,190 £ 
Cholecystectomy 1 9,6 1 0 £ 
Bascom 8, 1 70 £ 
270 new consultations 
Mean three conditions 



84 days before 
28 days after 



20 
26 
88 
20 
88 

1,299 
1,541 

3,535 

5,076 



88 
39 



-I9 C 

1,299 

1,408 

1,859 

3,267 



133 
504 
1809 
37 



±25% 

1,156-1,926 



±25% 

3,807-6,345 



±25% 

1,056-1,760 



±25% 

2,451-4,083 

100-167 
401-670 
1,357-2,261 
29-49 



Notes: Estimated for 1,000 patients surgically treated for selected surgical conditions. One-stop weight: 0.27 x 1,000 patients -270 one-stop 
20% of the patients were on paid sick leave; b the estimated mean for inguinal hernia, sinus pilonidalis, and laparoscopic cholecystectomy. The 
number of OSS patients needed to establish a cost-effective service. Estimated saving per patient due to decreased outpatient consultations. 
Abbreviations: C/E, cost-effectiveness; H, hospital perspective; min, minute; pt, patient; S, societal perspective; OSS, one stop surgery. 



patients. We assumed that 
threshold is defined as the 




Large-scale implementation 



5,153 



8,588 



I 1 1 1 1 1 ' — I 1 1 1 1 1 

56,000 58,000 60,000 62,000 64,000 66,000 68,000 70,000 72,000 74,000 76,000 78,000 



Cost (£) 

Figure 2 Cost uncertainty of guideline implementation in the GP EMR. 

Note: The variables with the highest impact on cost are listed at the top and ranked thereafter. 
Abbreviations: CDS, clinical decision support; EMR, electronic medical record; GP, general practitioner. 
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care. 35 For hospital decision-makers, high-quality referrals 
and better communication between GPs and hospital special- 
ists might have large implications for the quality of care. 
Our hospital trust receives approximately 140,000 referrals 
annually, imposing a huge burden in terms of coordina- 
tion and the workload for physicians and administrative 
personnel. A CDS referral system can improve coordina- 
tion, decrease hospital travel time by improving process 
management, and reduce the burden on busy outpatient 
clinics. 27 Improved coordination has been shown to have 
large economic implications. 2 However, better technological 
coordination of EMR systems and greater standardization 
of patient pathways are needed. Our analyses show that the 
waiting time for surgery will decrease, thus decreasing the 
burden for patients and resulting in cost savings for society as 
the duration of sick leave decreases. Finally, fewer resources 
will be used by admitting hospitals because of the improve- 
ments in logistics and the coordination of care. 

Conclusion 

As referral rates and reports of poor coordination between 
primary and secondary care increase, quality improvement 
projects focusing on referral quality are needed. 2 - 2 ''- 31 CDS 
systems have the potential to improve communication and 
coordination between primary and secondary care and thus 
increase the OSS rate. 32 This study has two main conclu- 
sions. First, many resources (personnel, time, and funding) 
were needed to implement a CDS-supported referral service, 
mainly because of the time-consuming process of establishing 
a medical consensus, unexpected technology obstacles, and 
organizational obstacles in the interface between primary and 
secondary care. Increased efforts are needed to make CDS 
referral systems more attractive for GPs, and incentives must 
respond to the increase in the referral workload. 
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Table SI Details of the unit costs assigned to health care resource use data 


Variable 


Unit cost (£)* 


Sensitivity analyses 


Cost of travel 3 




±25% 


Mean costs of hospital travel 


88 




Cost of surgeon 




±25% 


Mean salary/hour b 


102 




Surgeon outpatient consultation, 30 minutes' 


69 




Cost of hospital administrative personnel 6 


40 




Cost of IT expert 6 


107 




Cost related to sick leave 




±25% 


Governmental reimbursement of 1 day work absence' 


83 




Cost related to surgery 




±25% 


Cost of hernia surgery 


1,1 19 




Cost of cholecystectomy 


1,961 




Cost of sinus pilonidalis 


817 





Notes: ^Exchange rate on June 29, 2012: I £ -9.36 NOK. hup://www.dnb.no/en/currencylist?la=EN&site=DNB_NO ; "personal communication (September I, 2010) North 
Norwegian Health Administration (jN): 828 NOK per travel -88 £ per travel; b local data: Based on a mean salary of 102 £ (965 NOK)/hour for a hospital physician, and a mean 
salary for administrative personnel of 40 £. Data from hospital administration; ^Norwegian Health Authorities. Reimbursement and DRG weighting in Norwegian Hospitals 
20 1 2: http://www.helsedirektoratet.no/publikasjoner/regelverk-innsatsstyrt-finansiering-20 1 2/Sider/default,aspx . I DRG weight: 38,209 NOK; Outpatient consultation (day and 
night-time): DRG 923, weight 0.0 1 7; Cost of hernia surgery: DRG 1 ,620, weight 0.275: I , I 1 9 £; Cost of cholecystectomy: DRG 4,940, weight 0,482: 1 ,96 1 £; Cost of sinus 
pilonidalis: DRG 1 ,580, weight 0,20 1 : 8 1 7 £; 'Estimated from a median income of 276,000 NOK/year/patient as reported by Statistics in Norway: http.7/www.ssb.no/english/ . 
Abbreviations: DRG, Diagnoses Related Groups; JN, Jan Norum; NOK, Norwegian Kroner; IT, information technology. 
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